Language model

A probability distribution over possible documents. See also LLMs.

It refers to statistical models of documents. Usually, it is about the conditional probability of the next word given the previous words seen so far.

$$\hat{P}\left(w_{1}^{T}\right)=\prod_{t=1}^{T} \hat{P}\left(w_{t} \mid w_{1}^{t-1}\right), %p(x)=\prod_{i=1}^{n} p\left(s_{n} \mid s_{1}, \ldots, s_{n-1}\right)$$

where $w_t$ is the $t$‘th word in the sequence and $w_{i}^{j}$ is the subsequence $\left(w_{i}, w_{i+1}, \cdots, w_{j-1}, w_{j}\right)$.

If we have a good language model, we can do Natural language generation.

Powerful language models can generate human-like contents, which can poses various risks. See Risks of language models.

Natural Language Processing | Natural language understanding < > LLMs